CWE-230 Improper handling of missing values #947

dwiley258 · 2025-07-22T11:03:08Z

No description provided.

myteron

few cosmetics, +1 otherwise.

myteron · 2025-07-24T10:56:24Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+## Non-Compliant Code Example
+
+This noncompliant code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.


Suggested change

This noncompliant code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.

The `noncompliant01.py` code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.

myteron · 2025-07-24T10:56:55Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+## Compliant Solution
+
+The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.


Suggested change

The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.

The `compliant01.py` the method `Decimal.quantize` is used to gain control over known rounding errors in floating point values.

The sentence also needs a small rewording. Maybe something like this?

Suggested change

The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.

In the `compliant01.py` cod example, the method `Decimal.quantize` is used to gain control over known rounding errors in floating point values.

s19110

Finished the review. The code overall looks good, I had only one small comment to it, but there were also some small problems in the readme.

s19110 · 2025-07-24T13:51:09Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

@@ -0,0 +1,148 @@
+# CWE-230: Improper Handling of Missing Values
+
+In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered. The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html) Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.


Since we try to have one summary sentence in the first paragraph for the search engines, perhaps something like this could work?

Suggested change

In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered. The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html) Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.

The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html).

In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered. Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.

s19110 · 2025-07-24T14:01:32Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+## Compliant Solution
+
+The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.


The sentence also needs a small rewording. Maybe something like this?

Suggested change

The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.

In the `compliant01.py` cod example, the method `Decimal.quantize` is used to gain control over known rounding errors in floating point values.

s19110 · 2025-07-24T14:01:58Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.
+
+The decision by the balance_is_positive method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.


Suggested change

The decision by the balance_is_positive method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.

The decision by the `balance_is_positive` method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.

s19110 · 2025-07-24T14:04:17Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+`Decimal` throws a `decimal.InvalidOperation` for `NaN` values, the controlled rounding causes only `"0.01"` to return `True`.
+
+In `compliant02.py` we use the math.isnan to very if the value passed is a valid `float` value.


Suggested change

In `compliant02.py` we use the math.isnan to very if the value passed is a valid `float` value.

In `compliant02.py` we use the `math.isnan` to verify if the value passed is a valid `float` value.

s19110 · 2025-07-24T14:06:06Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+This behavior is compliant with IEEE 754[[2024 Wikipedia]](https://en.wikipedia.org/wiki/IEEE_754) a hardware induced compromise.
+The [example01.py](example01.py) code demonstrates various comparisons of `float('NaN')` all resulting in `False`.
+
+```python


Suggested change

```python

```python

# SPDX-FileCopyrightText: OpenSSF project contributors

# SPDX-License-Identifier: MIT

s19110 · 2025-07-24T14:06:31Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+*[noncompliant01.py](noncompliant01.py):*
+
+```python
+""" Non-compliant Code Example """


Suggested change

""" Non-compliant Code Example """

# SPDX-FileCopyrightText: OpenSSF project contributors

# SPDX-License-Identifier: MIT

""" Non-compliant Code Example """

s19110 · 2025-07-24T14:10:53Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/compliant02.py

+    _value = float(value)
+    if math.isnan(_value) or _value is None:
+        raise ValueError("Expected a float")
+    if _value <= 0:


This might be slightly confusing because print(balance_is_positive("0.001")) returns false in compliant01.py but returns true here. If we assume the balance uses cents, we could just change the threshold in this condition:

Suggested change

if _value <= 0:

if _value < 0.01:

If we do so, the same should be done in noncomplaint01.py so that this part of the code remains unchanged.

s19110 · 2025-07-24T14:15:02Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+|Tool|Version|Checker|Description|
+|:----|:----|:----|:----|
+|Bandit|1.7.4 on Python 3.10.4|Not Available||
+|flake8|flake8-4.0.1 on python 3.10.4||FS002 '.format' used|


I run flake8 on my machine and no error was shown. I don't see .format used in any code example, so I assume it's a leftover from an old version of the rule.

Suggested change

|flake8|flake8-4.0.1 on python 3.10.4||FS002 '.format' used|

|flake8|flake8-4.0.1 on python 3.10.4|Not Available||

s19110 · 2025-07-24T14:20:30Z

docs/Secure-Coding-Guide-for-Python/CWE-703/CWE-230/README.md

+
+|||
+|:---|:---|
+|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)|[IDS06-J. Exclude unsanitized user input from format strings](https://wiki.sei.cmu.edu/confluence/display/java/IDS06-J.+Exclude+unsanitized+user+input+from+format+strings)|


This SEI CERT rule seems unrelated. I have found another one that talks specifically about NaN values:

Suggested change

|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)|[IDS06-J. Exclude unsanitized user input from format strings](https://wiki.sei.cmu.edu/confluence/display/java/IDS06-J.+Exclude+unsanitized+user+input+from+format+strings)|

|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)|[NUM07-J. Do not attempt comparisons with NaN](https://wiki.sei.cmu.edu/confluence/display/java/NUM07-J.+Do+not+attempt+comparisons+with+NaN)|

Before this commit the wording was that modifying list works but is not recommended. But it works as long as no two consecutive elements are deleted, otherwise part of elements is not checked at all without any exceptions raised. Changed README.md, compliant01.py and noncompliant01.py to demonstrate that. Signed-off-by: Kyrylo Yatsenko <[email protected]> Signed-off-by: Helge Wehder <[email protected]> Signed-off-by: ewlxdnx <[email protected]>

…f#936) * guide Signed-off-by: balteravishay <[email protected]> * guide Signed-off-by: balteravishay <[email protected]> * remove temp files Signed-off-by: balteravishay <[email protected]> * lint Signed-off-by: balteravishay <[email protected]> * Update docs/Security-Focused-Guide-for-AI-Code-Assistant-Instructions.md Signed-off-by: Avishay Balter <[email protected]> * Update docs/Security-Focused-Guide-for-AI-Code-Assistant-Instructions.md Signed-off-by: Avishay Balter <[email protected]> --------- Signed-off-by: balteravishay <[email protected]> Signed-off-by: Avishay Balter <[email protected]> Signed-off-by: ewlxdnx <[email protected]>

Signed-off-by: ewlxdnx <[email protected]>

Signed-off-by: Helge Wehder <[email protected]> Signed-off-by: ewlxdnx <[email protected]>

dwiley258 force-pushed the CWE-230 branch from 9957bc5 to 1f9baa7 Compare July 22, 2025 11:07

myteron added the Product: Python Hardening Guide label Jul 24, 2025

myteron reviewed Jul 24, 2025

View reviewed changes

s19110 suggested changes Jul 24, 2025

View reviewed changes

hedrok and others added 6 commits July 31, 2025 11:35

CWE-230 Improper handling of missing values

3a4f627

Signed-off-by: ewlxdnx <[email protected]>

Added updates from comments

3189a0e

Signed-off-by: ewlxdnx <[email protected]>

fixed some linting issues to let PR review go ahead

2825490

Signed-off-by: Helge Wehder <[email protected]> Signed-off-by: ewlxdnx <[email protected]>

some cosmetics

f744186

Signed-off-by: Helge Wehder <[email protected]> Signed-off-by: ewlxdnx <[email protected]>

dwiley258 force-pushed the CWE-230 branch from d738b70 to f744186 Compare July 31, 2025 10:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CWE-230 Improper handling of missing values #947

CWE-230 Improper handling of missing values #947

Uh oh!

dwiley258 commented Jul 22, 2025

Uh oh!

myteron left a comment

Uh oh!

myteron Jul 24, 2025

Uh oh!

myteron Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 left a comment

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

s19110 Jul 24, 2025

Uh oh!

Uh oh!


		## Non-Compliant Code Example

		This noncompliant code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.

	This noncompliant code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.
	The `noncompliant01.py` code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with `NaN` in `_value == float("NaN")`.


		## Compliant Solution

		The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.

	The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.
	The `compliant01.py` the method `Decimal.quantize` is used to gain control over known rounding errors in floating point values.

	The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.
	In the `compliant01.py` cod example, the method `Decimal.quantize` is used to gain control over known rounding errors in floating point values.

		@@ -0,0 +1,148 @@
		# CWE-230: Improper Handling of Missing Values

		In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered. The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html) Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.

-In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered. The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html) Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.
+The `NaN` value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html).
+In python, some datasets use `NaN` (not-a-number) to represent the missing data. This can be problematic as the `NaN` values are unordered.  Any ordered comparison of a number to a not-a-number value are `False`. A counter-intuitive implication is that `not-a-number` values are not equal to themselves.


		The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values.

		The decision by the balance_is_positive method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.

	The decision by the balance_is_positive method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.
	The decision by the `balance_is_positive` method is to `ROUND_DOWN` instead of the default `ROUND_HALF_EVEN`.


		`Decimal` throws a `decimal.InvalidOperation` for `NaN` values, the controlled rounding causes only `"0.01"` to return `True`.

		In `compliant02.py` we use the math.isnan to very if the value passed is a valid `float` value.

	In `compliant02.py` we use the math.isnan to very if the value passed is a valid `float` value.
	In `compliant02.py` we use the `math.isnan` to verify if the value passed is a valid `float` value.

	\|flake8\|flake8-4.0.1 on python 3.10.4\|\|FS002 '.format' used\|
	\|flake8\|flake8-4.0.1 on python 3.10.4\|Not Available\|\|

	\|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)\|[IDS06-J. Exclude unsanitized user input from format strings](https://wiki.sei.cmu.edu/confluence/display/java/IDS06-J.+Exclude+unsanitized+user+input+from+format+strings)\|
	\|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)\|[NUM07-J. Do not attempt comparisons with NaN](https://wiki.sei.cmu.edu/confluence/display/java/NUM07-J.+Do+not+attempt+comparisons+with+NaN)\|

CWE-230 Improper handling of missing values #947

Are you sure you want to change the base?

CWE-230 Improper handling of missing values #947

Uh oh!

Conversation

dwiley258 commented Jul 22, 2025

Uh oh!

myteron left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s19110 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!